In distributed caching systems, especially in situations where the amount of cached data grows over time (such as with an online service that is continually adding new users), resharding or rebalancing the caching system is often needed to maintain an acceptable level of system performance. This causes at least two issues. First, resharding or rebalancing the caching system may require temporarily bringing the system offline while the resharding or rebalancing is performed. During the time the system is offline, read requests from applications that would otherwise have been served by the caching system are instead served from a backing data storage system, potentially creating catastrophic computing resource demand on the backing data storage system. Second, the in-place nature of the resharding or rebalancing is an issue. In particular, reverting the caching system to the pre-resharding and pre-rebalancing state may be time consuming or impractical. For example, reverting may require starting from an empty or cold cache or bringing the caching system offline again. Combined, these two issues make it difficult to effectively reshard or rebalance the cache since bringing the caching system temporarily offline or starting with an empty or cold cache is not practical in many circumstances because of the impact it causes on the computing resources of the backing data storage system.
Embodiments disclosed herein address these and other issues.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.